The relative change of treatment effect after adjustment for small study effects is investigated in this document. Three adjustment methods are applied: Trim and fill, copas selection model and the regression model. The estimates of this methods are compared to fixed effects and random effects meta-analysis estimates.

The relative change of treatment effect after adjustment is defined as:

\(1 - |{\frac{\theta_{C}}{\theta_{M}}}|\)

were \(\theta_{M}\) is the estimate of the meta-analysis and \(\theta_{C}\) is the estimate of the adjustment method and \(||\) denotes the absolute value.

The number can be interpreted as the fraction by which the original treatment effect is reduced after adjustment. If the number is positive, the treatment estimate has been reduced by the adjustment method. If the relative change is negative, then the adjusted treatment effect estimate is larger than the estimate of the meta-analysis

Trim-and-fill

First, the number of treatment effects that are smaller or larger than the treatment effects are counted:

## # A tibble: 1 x 2
##   biased larger.meta.effects
##    <dbl>               <int>
## 1    122                1018
## # A tibble: 1 x 2
##   biased smaller.meta.effects
##    <dbl>                <int>
## 1     32                  290
## # A tibble: 1 x 2
##   biased smaller.meta.effects.random
##    <dbl>                       <int>
## 1     35                         394

The empirical distribution of relative effect changes are shown in the following plots. Different ranges and different plots are shown for random and fixed effects meta-analysis and continuous and binary outcomes. Additionally, the fraction of meta-analyses with significant peters publication bias test (binary outcomes) and eggers test (continuous outcomes) is color-coded within the histogramms.

Binary Outcomes

The absolute values of the treatment effect estimates can be plotted against each other. This is shown, again for different meta-analysis methods.

Continuous Outcomes

metac.cont %>% mutate(xx = 1 - abs(est.trimfill.fixef.cont/est.fixef.cont)) %>% filter(xx > 0) %>% ungroup() %>% 
  summarise(biased = sum(egger.test), larger.meta.effects = n())
##   biased larger.meta.effects
## 1     34                 163
metac.cont %>% mutate(xx = 1 - abs(est.trimfill.fixef.cont/est.fixef.cont)) %>% filter(xx < 0) %>% ungroup() %>% 
  summarise(biased = sum(egger.test), smaller.meta.effects = n())
##   biased smaller.meta.effects
## 1      7                   57
metac.cont %>% mutate(xx = 1 - abs(est.trimfill.fixef.cont/est.ranef.cont)) %>% filter(xx < 0) %>% ungroup() %>% 
  summarise(biased = sum(egger.test), smaller.meta.effects.random = n())
##   biased smaller.meta.effects.random
## 1      6                          78

The absolute values of the treatment effect estimates can be plotted against each other. This is shown, again for different meta-analysis methods.

Copas Method

The structure of the example of trim-and-fill is maintained:

Binary Outcomes:

## # A tibble: 1 x 2
##   biased larger.meta.effects
##    <dbl>               <int>
## 1     96                 379
## # A tibble: 1 x 2
##   biased smaller.meta.effects
##    <dbl>                <int>
## 1     26                  101
## # A tibble: 1 x 2
##   biased smaller.meta.effects.random
##    <dbl>                       <int>
## 1     22                         108

Continuous Outcomes:

##   biased larger.meta.effects
## 1     26                  70
##   biased smaller.meta.effects
## 1     11                   33
##   biased smaller.meta.effects.random
## 1      9                          24

Regression

Binary Outcomes:

## # A tibble: 1 x 2
##   biased larger.meta.effects
##    <dbl>               <int>
## 1    105                 899
## # A tibble: 1 x 2
##   biased smaller.meta.effects
##    <dbl>                <int>
## 1     49                  409
## # A tibble: 1 x 2
##   biased smaller.meta.effects.random
##    <dbl>                       <int>
## 1     42                         407

Continuous Outcomes:

##   biased larger.meta.effects
## 1     27                 191
##   biased smaller.meta.effects
## 1     14                  103
##   biased smaller.meta.effects.random
## 1      8                          83